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ABSTRACT 



This study was conducted to analyze student background 
characteristics and educational history data that affect participation and 
performance of non-English language background students, including those with 
limited English proficiency, on Minnesota's Basic Standards Test. Data were 
obtained from individual student cumulative files in six schools in two 
districts with large numbers of non-English language background students. The 
cumulative files of more than 600 students were examined, but many (17% and 
19% in the 2 districts respectively, with up to 100% at 1 school) were not 
usable. The review of student files resulted in the development of 
recommendations for better data collection, with the ideal being a 
computerized database for each school and district to eliminate duplication 
of records. Also needed are commonly recognized data elements to be collected 
for all English language learners. Selecting computer software to make 
sharing information easier is also recommended. Other recommendations center 
on the use of standardized tests and training for school staff. (Contains 10 
references.) (SLD) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



TM030826 



i 



» 



00 

CN 

o 



^r 



Q 



W 



Linking student and programmatic characteristics to test 
performance: Issues and solutions 



Kristin Kline Liu, Minnesota Assessment Project, 

Martha Thurlow, National Center on Educational Outcomes 
Debra Albus, Minnesota Assessment Project 
Richard Spicuzza, Minneapolis Public Schools 
Jane Minnema, National Center on Educational Outcomes 

Paper prepared for the annual meeting of the American Educational Research 
Association, April 24-28, 2000, New Orleans: LA. 



Preparation of this paper was supported in part by the U.S. Department of Education, 
Office of Educational Research and Improvement (Grant #R279A501 1). Points of view 
or opinions expressed in the paper are not necessarily those of the Department or the 
Offices within it. 



BEST COPY AVAILABLE 






U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

□ This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 

\. ... ^ 





O 

ERIC 



2 



K. Liu 

AERA 47.32 



Two pieces of federal legislation, Title I of the Improving America’s Schools Act 
and the National Education Goals, require that data be collected on the academic 
performance of at-risk students to measure their progress toward meeting high standards. 
According to Title I law local education agencies must “Obtain accurate information 
about the academic progress of schools, and about the progress of LEP [limited English 
proficient] and ‘poor’ children within these schools...” (August, Hakuta, Olguin and 
Pompa, 1995). Schools must then use the data to make school improvement decisions. 

While the majority of states, districts and schools in the country are implementing 
some type of high standards, schools and districts are having difficulty collecting data 
and maintaining data files on which to base these instructional decisions (Far West 
Laboratory for Educational Research and Development, 1995). Collecting data on LEP 
students can be especially challenging. 

One of the largest obstacles to collecting data on limited English proficient 
students (also known as English language learners or ELLs) is the high mobility rate of 
the students and the resulting stress that a highly mobile population places on schools. 
Kerbow (1996) urges educators and policymakers to view mobility as a central issue in 
reform initiatives. “Typically, “ Kerbow says, “student mobility and the resulting school 
instability are relegated to the status of a background condition— that is, they are viewed 
as part of an external context to which schools must adjust. But mobility’s effects can be 
deep and pervasive, penetrating the essential activity of schools— the interaction of 
teachers and students around learning. Moreover, not only does mobility affect those 
students who are changing schools, it also more generally disrupts the functioning of 
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classrooms and the basic operations of schools (p. 163).” When there is a high mobility 
rate in a school: 

• It is difficult to do longitudinal monitoring of a group of students when the 
membership in the group changes significantly over the school year (Kerbow, 1996; 
Asher, 1991) 

• There are increased amounts of record keeping for school staff and possibly a 

/ 

decrease in staff commitment to collect data on students who may only be in the 
school for a short time (Asher, 1991) 

/ • The time needed for student cumulative records, the “permanent” files that travel with 
a student, to arrive with a transfer student impacts the new school’s ability to make 
appropriate placement decisions (Kerbow 1996; Nevada State Department of 
Education, 1996; Asher, 1991). Students may be the primary sources of information 
about their previous experiences, knowledge and skills (Lash and Kirkpatrick, 1990). 

• The teachers often bear most of the responsibility for integrating new students into 
the classroom, along with their other duties, and may be overwhelmed by the task if 
there are significant numbers of new students (Kerbow, 1996; Nevada State 
Department of Education, 1996; Lash and Kirkpatrick, 1990). Having a high 
percentage of transfer students coming in to the classroom at different points during 
the year may impact the teacher’s job satisfaction by making it difficult for teachers 
to plan cohesive instruction that will benefit students in the long-term (Kerbow, 1996; 
Lash and Kirkpatrick, 1990). In this kind of atmosphere teachers may feel less 
responsible for reading new student files and using the data to help make instructional 
decisions. Furthermore, data collection efforts at the school level often rely on 
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teachers to collect data and a teacher with a constantly changing population of 
students in his or her classroom may not be able to keep student files updated. 
Teachers are asking for help in collecting more and better data on highly mobile 
students (Lash and Kirkpatrick, 1990). 

When examining schools’ and districts’ ability to provide data for research on 
reform issues, clearly the impact of mobility upon the quality and availability of data 
must be considered. 

Purpose of the study 

This study was conducted to thoroughly analyze student background 
characteristics and educational history data that affect participation and performance of 
non-English language background students, including students with limited English 
proficiency, in Minnesota’s Basic Standards Tests (BSTs). The Basic Standards Tests are 
minimum competency tests in reading, mathematics and writing that students must pass 
by 12th grade in order to be eligible for a diploma. As the researchers collected and 
analyzed BST data and attempted to relate them to background and educational 
information on these students, observations about ways to improve data collection and 
storage were compiled. 



Methods/Data Sources 

A data collection form (see Appendix A) was created in cooperation with state 
education agency personnel who specialize in English Language Learners. Items to 
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include on the data sheet were drawn from a literature review, from data that districts 
submit to receive federal funding for English as a Second Language and Bilingual 

f 

programs, and from the experience of state education agency personnel. 

For Title I purposes, the federal government requires that large-scale assessment 
data be disaggregated by several categories (August, et al., 1995) including English 
proficiency, Special Education status, race and/or ethnicity, socio-economic status, 
migrant status and gender. However, for English language learners there are other ways 
to examine achievement data that may ultimately be more enlightening than simply 
examining achievement for all limited English proficient students as a group. In a study 
of the National Assessment of Educational Progress (NAEP), Abedi, Lord and Hofstetter 
(1998) found that data on length of time in the United States, length of time of academic 
schooling in the United States, and mobility are strongly related to the academic 
achievement of English language learners. These data elements were included on the 
data collection form. 

Data collection forms were designed to be used with student cumulative files since these 
files are the “permanent” records that travel with students when they change schools. 

Data were obtained from individual student cumulative files in six schools in two 
districts with large numbers of non-English language background students. There was 
one junior high and one senior high in each district, also an Alternative Learning Center 
(ALC) and a school for students with emotional-behavioral disabilities in one of the 
districts. One of the districts was a large urban district, and the other was a rural district. 
Cumulative file data were supplemented with data from ESL teacher files, district and 
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state databases, and other sources as needed. The cumulative files of more than 600 
students were examined. 



Findings/Results 

In some schools a sizeable portion of the cumulative files contained so little data 
that they were not useable in this study, while in other files particular data elements often 
conflicted with each other, were outdated or were simply left blank. Table 1 below 
shows the total number of cumulative files that were not useable at each school. 

Table 1: Cumulative Files not able to be Used 





Files 

reviewed 


Files not 
used 


% Files not 
used 


District A 








District 


137 


23 


17% 


Sr. High 


64 


3 


5% 


Jr. High 


47 


2 


4% 


ALC 


21 


13 


62% 


EBD Site 


5 


5 


100% 


District B 








District 


509 


97 


19% 


Sr. High 


390 


92 


24% 


Jr. High 


119 


5 


4% 



Overall, the percentage of cumulative files not able to be used in Districts A and 
B was similar (17% and 19% respectively), but this is largely due to the fact that District 
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A reviews included an Alternative Learning Center (ALC) and a site for students with 
emotional-behavioral disabilities (EBD) with high percentages of cumulative files that 
were not useable (62% and 100% of those reviewed). District B, the urban district, had 
the highest percentage of cumulative files that were not useable. In contrast, the Jr. High 
had a low percentage of unusable cumulative files. Junior high staff had obtained grant 
money to pay for a staff person to maintain files and, therefore, the quality of the records 
was quite good. 

Given the fact that the percentage of useable cumulative files varies greatly 
between schools and districts in this study, what factors can account for this variability? 

It has been well documented that students who are the most economically disadvantaged 
are also among the most mobile (see Asher, 1991 for more on this topic). Non-English 
language background students, a group including those with limited English proficiency, 
are among those students who have the lowest socio-economic status and the highest 
mobility rates. Table 2 shows each of the schools and districts in this study with relevant 
demographic information about the percentage of LEP students in the school, the 
percentage of students receiving free or reduced lunch, and both inter-district mobility (in 
and out of the district) and intra-district mobility (between schools in a district). These 
factors directly impact the quality of the data that were obtained 
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The table above shows that in schools with high percentages of limited English 
proficient students, higher mobility rates and higher concentrations of students receiving 
free or reduced lunch, the percentage of cumulative files not able to be used was also 
higher. Not only did District B have the highest percentage of cumulative files not 
useable, it also had one of the highest percentages of students receiving free or reduced 
lunch (65%-70%), and the highest overall percentage of LEP students (15%-16%). 

While the inter-district mobility rate in district B is lower than that in district A, it is still 
higher than the state average. The intra-district mobility rate (approximately 13%-27%) 
is 4 times to 6 times greater than the state average. Clearly the condition of the 
cumulative records is related to these factors. However, again, this was true with one 
exception — .the junior high in district B had files in good condition in spite of the fact that 
the school inter-district mobility rate is nearly 40% (three times the state average) the 
school intra-district mobility rate is roughly 30% (6 times to 10 times the state average), 
the percentage of LEP students is 15% to 16% (3 times to 5 times the state average) and 
that 75% to 80% of the student body receive free or reduced price lunches (3 or more 
times the state average). The state of the cumulative files in this junior high was directly 
related to the grant money that paid for their maintenance. 

The ALC and the school for EBD students in district A, the rural district, have 
the highest mobility rates, but they are schools that draw a unique group of students for a 
program specifically suited to their needs. Students transfer into those programs from 
other schools in the district and are also drawn from the surrounding areas. A placement 
in the ALC may be temporary in nature, designed to get students back into their original 
junior or senior high. Therefore, high mobility rates in these two programs may be 
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partially related to the nature of the programs. When looking at the percentage of files 
not able to be used at the alternative learning center and the school for students with 
emotional-behavioral disabilities, it is important to keep in mind that the total number of 
the student files reviewed was very small due to small school enrollments. 

Kerbow (1996) has suggested that schools examine the type of mobility patterns 
in their student body so that they can determine if a shared computer database of student 
records would be worthwhile. According to the data shown above, the schools in district 
B had roughly 15% to 30% of their student body transferring to and from schools within 
the same district. District A did not have high inter-district mobility rates, largely 
because the district has only one junior high and one senior high. It is not known whether 
districts A and B share any students who move between them. Therefore, district B 
schools would benefit from a shared database where student records could be exchanged 
rapidly via computer. In fact, the district does have such a database, but not all schools in 
the district are able to pay a trained staff person to operate the database and run reports. 

When cumulative files were able to be used, data were first taken from this 
source. If certain pieces of information were missing or unclear in the cumulative file, 
other sources were consulted. Since cumulative files are the “permanent” files all data 
found in these files were taken as true if data in other sources conflicted with data in 
cumulative files. Table 3 shows particular pieces of demographic data that were included 
in the data collection effort and how many of the total set of files consulted in each 
district did not contain specific pieces of data. This does not mean the data do not exist 
somewhere, they just were not found in the files examined at the time of the study. 
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Table 3: Missing data for each demographic data element 





District A (rural) 

n=114 


District B (urban) 

n=412 




Files 

without data 


% 


Files 

without data 


% 


Gender 


10 


9% 


6 


2% 


Home lang. 


6 


6% 


155 


38% 


Grade 


2 


2% 


0 


0% 


Graduation 

Year 


1 


<1% 


43 


10% 


Time in 
district 


31 


27% 


132 


32% 


School 

changes 


50 


44% 


305 


74% 


Years in U.S 
schools 


65 


57% 


241 


58% 


Years in 
U.S. 


55 


48% 


150 


37% 


Free/reduce 
d lunch 


4 


4% 


14 


3% 



*Percentages are rounded 



As shown in Table 2, time in the United States, time in U.S. schools, English 
proficiency levels and mobility are pieces of data that were found less often than pieces 
of demographic data in spite of the fact that these correlate highly with academic 
achievement. In district B, the urban district where the two schools had high percentages 
of limited English proficient students, high mobility rates and high percentages of 
students receiving free or reduced lunch, a basic data element such as a student’s home 
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language was missing from the files about 40% of the time on the day(s) data were 
collected. 

Tables 4 and 5 show pieces of language proficiency test data that were missing. 
This information was not kept in cumulative files. The data came from ESL teacher 
reports in district A and from the intake center for new students in District B. Typically, 
it is only limited English proficient students needing ESL or Bilingual services who are 
given a language proficiency test. District A gives the Woodcock Language Proficiency 
Battery, Revised Version (WLPB-R), which is a test normed on native speakers of 
English that measures academic English abilities. District B gives the Language 
Assessment Scale (LAS) to students who are new to the district. 

Table 4: Missing Data for Language Proficiency Tests in District A 





District A (rural) 
n=114 




Files 

without data 


% 


Woodcock 

Rdg 


35 


31% 


Woodcock 

Writing 


37 


32% 


Woodcock 

Broad 

English 


40 ~ 


35% 
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Table 5: Missing Data for Language Proficiency Tests in District B 





District B (urban) 




n=412 




Files 

without data 


% 


LAS Oral 
Score 


221 


54% 


LAS Oral 
Rank 


205 


50% 


LAS 

Rdg/Wrtg 

Score 


not given 
until student 
is level 4 
oral (fluent) 


not given 
until student 
is level 4 
oral (fluent) 


LAS 

Rdg/Wrtg 

Rank 


not given 
until student 
is level 4 
oral (fluent) 


not given 
until student 
is level 4 
oral (fluent) 



As seen in Table 4, about 30%-35% of District A’s non-English language 
background students do not have a Woodcock score in their files. This may indicate that 
they are not considered limited English proficient and have not been given the test 
because they are mainstreamed, or it may be that data are missing from files. 

District B only gives the oral test of the Language Assessment Scale (LAS) to 
incoming students who are new to the district. District B policy is that if the student 
receives a “fluent” oral score (level four out of five possible levels), then they are given 
the reading/writing test. Since LEP students typically score at levels 1 and 2 on the LAS 
oral test, there is a lack of data on the LAS reading/writing test in district B. There are 
three possible reasons why fifty to fifty-five percent of District B’s students do not have a 
LAS score on file. First, students may have transferred into the junior or senior high 
school in this study from other schools in the district and so were not given the LAS at 
the time of transfer. Second, some of the students in the study are Non-English language 
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background students who are not limited English proficient. Third, data may simply be 
missing from files. In district A, there are a number of students who move back and forth 
between Minnesota and a southern state. Time spent each year in a different school 
district may account for the fact that roughly 20 % of English language learners in District 
A have a LAS score of some kind in their files even though District A does not give the 
LAS. Table 3 clearly shows that in both districts A and B, while the LAS Oral Rank and 
Oral Score should both be found in a student’s file, often only one of these data elements 
is present. 

Other Problems in Data Collection 

In addition to the problem of missing data, data from different sources often 
disagreed, making it difficult to determine which data were accurate. This happened for 
several reasons: 

• Individual student files and student records in computer databases were updated at 
different times. Sometimes data were updated on a seemingly random schedule, so 
that data for one student might be several weeks or months older than data for another 
student 

• Files that had useable data often had too much data kept in them, making it difficult 
to sort through all of the papers 

• Since few school staff ever examined entire cumulative files, they may not be aware 
of conflicting data in the files 

• Parents and students sometimes provided varying information about home language, 
date of birth, etc. These variations could be due to low-level English or native 
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language skills, lack of understanding, unwillingness to provide data that seemed too 
personal, or simply because the information was unknown (e.g., refugees from a 
culture that did not record birth dates sometimes have to create one for student files) 

• Individual staff keep their own records and some data in those records do not get 
recorded in cumulative files (e.g., dates and results of English language proficiency 
assessments) 

• Data from multiple records may be transferred inaccurately from one location to the 
next. In addition, the route from one school to another and to the district database is 
often very complicated, opening up a wide range of places for data to be misplaced or 
misentered. As an example, Figure 1 shows the flow of Data in District B and how 
many times some of the pieces of information collected in this study had changed 
hands. All pieces of data should flow into the cumulative file, but some do not make 
it there. 
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Recommendations 

Based on the results of this study and the results of the literature review, the 
following recommendations are given to improve the quality of, and access to, data on 
students with limited English proficiency: 

Ideal 

1. The ideal to work toward is a computerized database for each school and district in a 
state that would eliminate the need for multiple sets of files with similar types of 
information and allow for the ability to rapidly share files with other schools when 
students transfer. This would be a different database from the one kept by the state and 
could be accessed over the internet, as some school districts are already doing (Quest, 
2000). Having such a database would take a large financial commitment to provide 
compatible computer equipment and software for each school, trained staff to enter data 
and maintain the database and personnel to collect the data. The results, however, could 
significantly strengthen teachers’ ability to provide high quality standards-based 
education for English language learners and to show that the school and district are 
accountable for educating these students. 

There are existing databases of student records that schools can look to as 
examples. One of the most widely implemented databases for tracking students was the 
Migrant Student Transfer System (MSTS), which has been phased out and replaced by 
the New Generation System, both developed by the Migrant Education Program of the 
federal government. Both systems were designed to collect and transfer education and 
health-related data on migrant students throughout the country. The Migrant Student 
Transfer System was a centralized database, while the New Generation System that is 
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currently in use is a web-based database. As schools and districts work toward the ideal 
of a shared computer database for English language learners, problems already existing 
databases provide helpful guidance on how to design a system teachers can use. 
According to Cahape (1993) and Asher (1991), problems with MSTS included: (1) 
inclusion of data elements for both local level and state level analyses overwhelmed data 
collectors and made them feel that there was no local use for many of the pieces of 
information collected. This, in turn, influenced their willingness to collect the data; (2) 
The system still relied on creation of paper documents before data were entered into the 
computer, creating multiple levels of reporting that resulted in delays in transferring data; 
(3) Lack of computer terminals connected to MSTS limited teachers’ ability to use the 
system to obtain information for use in making educational decisions. Instead, those who 
used MSTS often used it to validate decisions that were already made; (4) Data collection 
methods were not standardized across states and this impacted educators’ ability to use 
the data; (5) Migrant students and their parents did not have a well defined role in the 
system. As students moved between states they needed to notify new schools of their 
system identification number so that the new school could access student records. 
However, families were not always aware of their numbers or of the benefits to the 
student of having the MSTS. With the New Generation System, the database has been 
put on the Internet so that any pre-authorized person can access the database from a 
computer with a web browser. Having data on-line decreases the cost of transferring 
records, but the records are much less private (Asher, 1991). 
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2. Along with a shared database an ideal for states, districts and schools to work toward 
would be a commonly agreed upon set of data elements that are collected for all English 
language learners at the school and district level. The State Department of Education 
should work with schools and districts to facilitate conversation about what data elements 
are most beneficial for schools to collect for their own analyses. The National Center for 
Educational Statistics (2000) has created a data handbook with lists of recommended data 
elements and definitions that, if followed, would help data collection to become more 
standardized. Based upon the results of this study, the following pieces of data should be 
included for English language learners: Time in U.S., Time in U.S. schools, number of 
school changes, whether the student is identified as limited English proficient according 
to state criteria and would be eligible to receive ESL or Bilingual services (some who are 
eligible do not receive them) and the type of services, ESL or bilingual) a student 
receives along with an original starting date for those services. Each data element should 
have one agreed upon definition and one common operational definition that specifies 
how to record the data for that element. Having common definitions of data collected in 
the same manner would make it much easier for schools to interpret files they receive 
when students transfer and for outside researchers to use data from multiple schools in 
creating large data sets. Even if computer databases are not currently an option for many 
schools and districts, having an agreed upon set of terms and definitions would greatly 
enhance the quality of paper cumulative files as well. Keeping the number of data 
elements to a minimum is crucial so that school staff charged with collecting data are not 
overwhelmed by the task. 
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Steps Toward the Ideal 



3. As a step toward a large database shared between many school districts, schools and 
districts should investigate the record keeping capabilities of their current computer 
software. Microsoft’s Office 98 and Office 2000 come with a program called “Access” 
that can store a large amount of data and create reports (see 

http://www.athree.com/acess_info). Using Access, schools can create a simple database 
of the most important information on English language learners and create lists of 
students who meet particular criteria (Liu, 2000; Spicuzza, 2000). For example, if a 
school wants to examine the academic test scores of all students who speak Somali and 
have moved more than 3 times the software can easily create a list of these students. 

More sophisticated analyses require a more advanced type of database software such as 
Microsoft SQL Server (see http://www.microsoft.com/sql for more information). 
However, Access is part of a relatively common and inexpensive software package (cost 
about $340). With a limited amount of training, school staff who are comfortable using a 
computer could create a database that could be linked with others. For example, the ESL 
teacher could link her database to the database of student records kept by the main office 
in her school and reduce the need for both to have paper files with duplicate information. 
Student records in Access can be imported into Excel for more detailed reports and e- 
mailed to other parties who request access to data. 

Three of the major drawbacks to using Access are: (1) That entering a large 
amount of data into the database slows down the time required for processing a report 
(Liu, 2000); and (2) That anyone with access to the system can alter data. For that 
reason, it is recommended that there be a person charged with maintaining the database 
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and creating reports for other people. Changes made to data found in individual reports 
would not affect the data in the master database (Liu, 2000); (3) That the system is not 
designed to be used by more than a few users at the same time 
(http://www.athree.com/access_info/pros_and_cons.html) 

There are other software products that can create simple databases, such as 
Filemaker Pro (for more information see http://www.filemaker.com/filemaker.html). The 
researchers are not endorsing any particular piece of software. Something that is widely 
available would be preferable. 

4. For any type of student files, computerized or paper, schools and districts should keep 
a list of commonly used acronyms and abbreviations with each file. For example, there 
are many different terms that can be used to refer to English language learners (ELLs) 
and schools vary in which ones they use— limited English proficient (LEP), fully English 
proficient (FEP), non-English language background (NELB), language minority (LM), 
persons whose language is other than English (PHLOTE), English as a Second Language 
(ESL) student, etc. It is difficult for data users to interpret the data if there are no 
explanations of the terms. 

5. Schools should strive, as much as possible, to give standardized tests uniformly- the 
same sections of the tests, the same form (long vs. short) at the same ages to provide the 
most useable achievement data. Within one school it is difficult to find a large enough 
group of students with similar types of assessment data in order to do statistical analyses 
such as the correlation between a language proficiency test and a state graduation test. If 
educators would like to use data to advocate for different programming or for an 
alternative assessment for English language learners the assessment data on students 
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needs to be comparable across students. Schools should consider giving a language 
proficiency test to English language learners on a regular basis rather than just once to 
those students who are new to the district. 

6. School staff who collect data should fill out forms or fill in fields in a database 
completely, even if answers seem obvious. For example, many times the student’s 
primary language is not found in cumulative files. It might be easy to look at a student’s 
name and guess what primary language he or she speaks, but such a guess might not be 
accurate. A person with a Chinese name, for example, could have been bom in Central 
or Latin America and speak Spanish as a first language. Filling in forms completely, 
using “NA” for not applicable, giving both a raw score and a scale score for tests, if the 
form asks for both, would greatly improve the quality and usefulness of data. It would 
also make it easier for errors in data recording to be detected. 

7. School staff who work with data and who do not have a background in English as a 
Second Language or Bilingual Education should receive training in basic terminology 
used to refer to these students. The quality of data and of reports based on those data are 
related to the ability of staff to make distinctions between closely related and sometimes 
overlapping terms. For example, if a district data office is requested to print a report of 
all students who are “language minority”, those staff should be familiar with the term and 
know how it is different from the term “minority students”. If a district creates a list of 
commonly used terms in that district it is much easier to make data requests using 
standard terminology. 
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8. Data kept in paper or computer files should be reviewed periodically to make sure it is 
still useful, to find conflicts and errors, to locate incomplete data elements and fill them 
in. This is done more easily if only essential pieces of information are kept in the 
student’s “permanent” record. In more than one of the schools in this study, files 
contained so much data that few staff members had ever examined every piece of 
information in the file from beginning to end. Therefore, staff may not have been aware, 
for example, that highly mobile students had multiple forms asking about their home 
language (one administered each time students transferred). Even though these forms 
were translated into the students’ native language and English, parents often completed 
the forms differently over time. One time the parent gave Vietnamese as the student’s 
home language, the next time English, and the third time both Vietnamese and English 
equally. This discrepancy led researchers to believe that parents may not have 
understood how the information would be used, may not have had sufficient literacy 
skills to read the home language forms, may have believed that filling out the forms in a 
particular way would be an advantage to their child, etc. Less data in the files would 
have made this particular discrepancy easier to detect and in the case of this study, would 
have made it easier for both the initial file reviewer and the second rater to find the same 
piece of data on a student. 

9. Schedules for updating paper files and especially computer databases should be posted 
in the area where the files are kept so that it’s clear to data users that some files may 
contain newer data than others (e.g., some files have the most recent statewide test scores 
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entered and others do not). This is particularly important in large urban and suburban 
districts where there are too many files to update them all at the same time. 

10. In each file or in the area where files are kept, districts should post an updated list of 
changes in school names and school reorganizations that result in different grade levels 
being served in a building from year to year. It is difficult to accurately count the number 
of school changes a student has had if it is not clear whether two different school names 
in a file represent a student transfer or the renaming of a school. School staff may not 
have been employed in the district long enough to know the history of all such changes. 

11. Efforts should be made to make the same data file accessible to parties with a 
legitimate use for the data. If a database exists in a school or district, often administration 
and program directors are the ones who have access to it and teachers, counselors and 
school psychologists use the cumulative file. (Outside researchers often do not know 
what information is kept in what location.) If the information in the database and the 
cumulative file duplicate each other, often the cumulative file is not maintained as well as 
the database and those with greatest need for current information are not receiving it. For 
example, in one of the districts studied, the district computer database had information on 
each student’s mobility, but this information was not in the cumulative file. Given that 
mobility has been found by many researchers to significantly impact student 
achievement, this is a piece of information that educators, counselors and school 
psychologists should have access to. Furthermore, if only one computer with a district 
database is available for teachers in a large metropolitan school, data will not be used to 
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inform decisions at the classroom level. Federal and State Departments of Education 
should consider giving funding for computers so school accountability data that are 
collected can be more easily accessed and used by teachers. 

12. School staff need training and support in how to use data for making educational 
decisions. In fact, their ability and willingness to collect data on individual students 
and maintain student files may depend upon their understanding of how that data can 
be used on a daily basis to the benefit of their students. State Departments of 
Education should consider providing such training to all types of staff, not just to 
administrators. 

13. Districts and schools should clearly indicate on the transcript which classes are 
considered ESL or Bilingual Education classes. For example, a course such as “6466 
Advanced Reading and Writing” is clearly an ESL class, but a course such as #6235 
Physical Science is not necessarily listed on a school transcript as being taught in 
Spanish. Often times, the titles of courses are too long to completely fit on 
transcripts and what appears is a course number and the first few words. This issue 
arose in the high school in district B. A review of course enrollments, found in the 
cumulative files, did not indicate that any students were taking classes considered 
ESL or Bilingual Education, but the course schedule listed 37 possible ESL or 
Bilingual content classes. In district B, the closest approximation researchers could 
obtain to whether a student was getting ESL or Bilingual services at the time they 
took the Basic Standards Tests was whether the district database listed a start date for 
ESL or Bilingual Services. 
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